Search for: All records

Recognizing if LLM output can be grounded in evidence is central to many tasks in NLP: retrieval-augmented generation, summarization, document-grounded dialogue, and more. Current approaches to this kind of fact-checking are based on verifying each piece of a model generation against potential evidence using an LLM. However, this process can be very computationally expensive, requiring many calls to a model to check a single response. In this work, we show how to build small fact-checking models that have GPT-4-level performance but for 400x lower cost. We do this by constructing synthetic training data with GPT-4, which involves creating realistic yet challenging instances of factual errors via a structured generation procedure. Training on this data teaches models to check each fact in the claim and recognize synthesis of information across sentences. For evaluation, we unify datasets from recent work on fact-checking and grounding LLM generations into a new benchmark, LLM-AggreFact. Our best system MiniCheck-FT5 (770M parameters) outperforms all systems of comparable size and reaches GPT-4 accuracy. We release LLM-AggreFact, code for data synthesis, and models. 

There are thousands of unannotated translated open reading frames (ORFs) in the Saccharomyces cerevisiae genome. Previous investigation into one such unannotated ORF, which was systemically labeled YGR016C-A based on its genomic coordinates, showed that replacing the ORF's ATG start codon with AAG led to a change in cellular fitness under different stress conditions (Wacholder et al., 2023). This suggested translation of YGR016C-A plays a role in cellular fitness. Here, we investigate Ygr016c-a's subcellular localization to gain insight into its cellular function. Computational prediction tools, co-expression analysis and fluorescence microscopy suggest that the Ygr016c-a protein localizes to the endoplasmic reticulum. 

A non-aqueous proton electrolyte is devised by dissolving H3PO4 into acetonitrile. The electrolyte exhibits unique vibrational signatures from stimulated Raman spectroscopy. Such an electrolyte exhibits unique characteristics compared to aqueous acidic electrolytes: 1) higher (de)protonation potential for a lower desolvation energy of protons, 2) better cycling stability by dissolution suppression, and 3) higher Coulombic efficiency owing to the lack of oxygen evolution reaction. Two non-aqueous proton full cells exhibit better cycling stability, higher Coulombic efficiency, and less self-discharge compared to the aqueous counterpart.